A study on driving performance and In Vehicle infotainment system has been done. The study aims two answer two questions. 1) Can the addition of tactile navigation improve driving performance 2) Which tactile navigation (if any) improves driving performance most
The first part of this statistical analysis aims to investigate the sample to answer if the data is normally distributed and if the participants’s characteristics are distributed across the four conditions. If this is not the case an effect of condition could be attributed to other personal factors of the participants (e.g. gaming experience).
Dependent variables:
- SUS
- SART
- Total Crash
- Lane Breaks
- Total Glances
- Total Glance Time
Independent variable:
- Condition (Acrylic, Bar, Combination & Touch only)
Demographics:
- Gender
- Gaming
- Drivers licence Years
- Automatic gear experience
- Car Simulator
A summary of ‘Age’ and ‘Years with drivers licence’. As with the bar graph for age we have a lot of people between 20-23 and most participants have had a drivers licence for 3-5 years.
## Age YearDriversLicence
## Min. :20.0 Min. : 1.833
## 1st Qu.:21.0 1st Qu.: 3.479
## Median :22.0 Median : 4.417
## Mean :22.7 Mean : 4.778
## 3rd Qu.:23.0 3rd Qu.: 5.417
## Max. :34.0 Max. :16.000
## [1] 2.575191
## [1] 2.418032
To answer the first question of whether condition have an effect on the 6 metrics we’ll create a plot of means with confidence (95%) as whiskers. According to Geoff Cumming and Sue Finch if the data has the same standard deviation it is possible to gain insight into which data could be significantly different.
## Condition N SUS sd se ci
## 1 Acrylic 11 80.68182 11.12940 3.355642 7.476835
## 2 Bar 11 80.90909 11.36182 3.425717 7.632973
## 3 Combination 11 80.45455 12.23817 3.689946 8.221712
## 4 Touch_only 11 75.00000 14.23025 4.290582 9.560012
## Condition N SART sd se ci
## 1 Acrylic 11 15.00000 5.253570 1.584011 3.529396
## 2 Bar 11 14.72727 3.951985 1.191568 2.654979
## 3 Combination 11 14.54545 3.643175 1.098459 2.447519
## 4 Touch_only 11 14.00000 4.427189 1.334848 2.974226
## Condition N Total_crash sd se ci
## 1 Acrylic 11 0.5454545 0.9341987 0.2816715 0.6276032
## 2 Bar 11 0.6363636 0.8090398 0.2439347 0.5435204
## 3 Combination 11 0.4545455 0.5222330 0.1574592 0.3508409
## 4 Touch_only 11 0.5454545 0.9341987 0.2816715 0.6276032
## Condition N Lane_Breaks sd se ci
## 1 Acrylic 11 3.636364 3.074824 0.9270945 2.065695
## 2 Bar 11 4.454545 4.590504 1.3840890 3.083943
## 3 Combination 11 3.909091 2.625054 0.7914836 1.763535
## 4 Touch_only 11 3.818182 2.750207 0.8292185 1.847614
## Condition N Total_Glances sd se ci
## 1 Acrylic 11 45.27273 12.042350 3.630905 8.090160
## 2 Bar 11 47.18182 6.274045 1.891696 4.214961
## 3 Combination 11 55.00000 13.798551 4.160420 9.269992
## 4 Touch_only 11 45.27273 7.156688 2.157823 4.807928
## Condition N Total_Glance_time sd se ci
## 1 Acrylic 11 34.18545 10.471866 3.157387 7.035096
## 2 Bar 11 34.71636 7.799066 2.351507 5.239484
## 3 Combination 11 40.01091 10.989521 3.313465 7.382861
## 4 Touch_only 11 38.80273 10.009391 3.017945 6.724400
To test significance we will use ANOVA
ANOVA has Three assumptions
1) Normality
2) Homogenity
3) Independence
The assumption of Independence is meet as our participants haven’t
influence each other in any way we know of. The participants were
instructed not to speak with each other between trials and we can hope
they have met this instruction.
Cond1 <- subset(SartSusDemo, SartSusDemo$Condition=='Combination')
Cond2 <- subset(SartSusDemo, SartSusDemo$Condition=='Touch_only')
Cond3 <- subset(SartSusDemo, SartSusDemo$Condition=='Bar')
Cond4 <- subset(SartSusDemo, SartSusDemo$Condition=='Acrylic')
shapiro.test(Cond1$SUS)
##
## Shapiro-Wilk normality test
##
## data: Cond1$SUS
## W = 0.73925, p-value = 0.001505
shapiro.test(Cond2$SUS)
##
## Shapiro-Wilk normality test
##
## data: Cond2$SUS
## W = 0.91515, p-value = 0.2803
shapiro.test(Cond3$SUS)
##
## Shapiro-Wilk normality test
##
## data: Cond3$SUS
## W = 0.92941, p-value = 0.4049
shapiro.test(Cond4$SUS)
##
## Shapiro-Wilk normality test
##
## data: Cond4$SUS
## W = 0.87781, p-value = 0.09759
shapiro.test(Cond1$SART)
##
## Shapiro-Wilk normality test
##
## data: Cond1$SART
## W = 0.92912, p-value = 0.4021
shapiro.test(Cond2$SART)
##
## Shapiro-Wilk normality test
##
## data: Cond2$SART
## W = 0.83396, p-value = 0.02632
shapiro.test(Cond3$SART)
##
## Shapiro-Wilk normality test
##
## data: Cond3$SART
## W = 0.96283, p-value = 0.8064
shapiro.test(Cond4$SART)
##
## Shapiro-Wilk normality test
##
## data: Cond4$SART
## W = 0.98203, p-value = 0.9764
shapiro.test(Cond1$Total_Glance_time)
##
## Shapiro-Wilk normality test
##
## data: Cond1$Total_Glance_time
## W = 0.94149, p-value = 0.5381
shapiro.test(Cond2$Total_Glance_time)
##
## Shapiro-Wilk normality test
##
## data: Cond2$Total_Glance_time
## W = 0.92679, p-value = 0.3793
shapiro.test(Cond3$Total_Glance_time)
##
## Shapiro-Wilk normality test
##
## data: Cond3$Total_Glance_time
## W = 0.95557, p-value = 0.7155
shapiro.test(Cond4$Total_Glance_time)
##
## Shapiro-Wilk normality test
##
## data: Cond4$Total_Glance_time
## W = 0.91494, p-value = 0.2787
shapiro.test(Cond1$Total_Glances)
##
## Shapiro-Wilk normality test
##
## data: Cond1$Total_Glances
## W = 0.97154, p-value = 0.9016
shapiro.test(Cond2$Total_Glances)
##
## Shapiro-Wilk normality test
##
## data: Cond2$Total_Glances
## W = 0.84398, p-value = 0.03561
shapiro.test(Cond3$Total_Glances)
##
## Shapiro-Wilk normality test
##
## data: Cond3$Total_Glances
## W = 0.91694, p-value = 0.294
shapiro.test(Cond4$Total_Glances)
##
## Shapiro-Wilk normality test
##
## data: Cond4$Total_Glances
## W = 0.94695, p-value = 0.6053
shapiro.test(Cond1$Lane_Breaks)
##
## Shapiro-Wilk normality test
##
## data: Cond1$Lane_Breaks
## W = 0.95245, p-value = 0.6754
shapiro.test(Cond2$Lane_Breaks)
##
## Shapiro-Wilk normality test
##
## data: Cond2$Lane_Breaks
## W = 0.95583, p-value = 0.7188
shapiro.test(Cond3$Lane_Breaks)
##
## Shapiro-Wilk normality test
##
## data: Cond3$Lane_Breaks
## W = 0.84061, p-value = 0.03216
shapiro.test(Cond4$Lane_Breaks)
##
## Shapiro-Wilk normality test
##
## data: Cond4$Lane_Breaks
## W = 0.90516, p-value = 0.2135
shapiro.test(Cond1$Total_crash)
##
## Shapiro-Wilk normality test
##
## data: Cond1$Total_crash
## W = 0.64917, p-value = 0.0001052
shapiro.test(Cond2$Total_crash)
##
## Shapiro-Wilk normality test
##
## data: Cond2$Total_crash
## W = 0.64952, p-value = 0.0001063
shapiro.test(Cond3$Total_crash)
##
## Shapiro-Wilk normality test
##
## data: Cond3$Total_crash
## W = 0.75438, p-value = 0.002371
shapiro.test(Cond4$Total_crash)
##
## Shapiro-Wilk normality test
##
## data: Cond4$Total_crash
## W = 0.64952, p-value = 0.0001063
There are 8 scores which break normality
Condition 1 - SUS (0.001505)
Condition 2 - SART (0.02632)
Condition 2 - Total Glances (0.03561)
Condition 3 - Lane breaks (0.03216)
Condition 1 - Total Crash (0.0001052)
Condition 2 - Total Crash (0.0001063)
Condition 3 - Total Crash (0.002371)
Condition 4 - Total Crash (0.0001063)
Another way to present the crash data would be to score crashes pr
condition:
It can be seen on this graph that each condition has about the same
total crashes and some conditions have one or two participants
responsible for extra crashes.
To test for homogeneity we’ll use Levines test for the scores which aren’t normally distributed and Bartletts test for the scores which are normally distributed.
## Levene's Test for Homogeneity of Variance (center = median)
## Df F value Pr(>F)
## group 3 0.2143 0.8859
## 40
## Levene's Test for Homogeneity of Variance (center = median)
## Df F value Pr(>F)
## group 3 0.4316 0.7315
## 40
## Levene's Test for Homogeneity of Variance (center = median)
## Df F value Pr(>F)
## group 3 2.3109 0.09079 .
## 40
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Bartlett test of homogeneity of variances
##
## data: Total_Glance_time by Condition
## Bartlett's K-squared = 1.2274, df = 3, p-value = 0.7464
## Levene's Test for Homogeneity of Variance (center = median)
## Df F value Pr(>F)
## group 3 0.0907 0.9647
## 40
## Levene's Test for Homogeneity of Variance (center = median)
## Df F value Pr(>F)
## group 3 0.4612 0.7109
## 40
We see that the variance in metrics across the four conditions are not significantly different.
We now have two options
1) Compute an ANOVA because it is said to be somewhat robust against
violations of normal distribution
2) Compute a Welsh ANOVA (it is robust against normal distribution but
not unequal variance (heteroscedastic))
3) Compute a Kruskal-Wallis test - as our data has equal varience
(homoscedasticity) but is not normally distributed
We will choose option 3) and compute the Kruskal-Wallis test
Note: Total Glance Time is both normally distributed and has homoscedasticity and therefore an anova can be computed
SUS <- kruskal.test(SUS ~ Condition, data = SartSusDemo)
SART <- kruskal.test(SART ~ Condition, data = SartSusDemo)
Glances <- kruskal.test(Total_Glances ~ Condition, data = SartSusDemo)
Crashes <- kruskal.test(Total_crash ~ Condition, data = SartSusDemo)
LaneBreaks <- kruskal.test(Lane_Breaks ~ Condition, data = SartSusDemo)
GlanceTime <- aov(Total_Glance_time ~ Condition, data = SartSusDemo, var.equal = TRUE)
SUS
##
## Kruskal-Wallis rank sum test
##
## data: SUS by Condition
## Kruskal-Wallis chi-squared = 1.973, df = 3, p-value = 0.578
SART
##
## Kruskal-Wallis rank sum test
##
## data: SART by Condition
## Kruskal-Wallis chi-squared = 0.23728, df = 3, p-value = 0.9714
Glances
##
## Kruskal-Wallis rank sum test
##
## data: Total_Glances by Condition
## Kruskal-Wallis chi-squared = 4.2639, df = 3, p-value = 0.2343
Crashes
##
## Kruskal-Wallis rank sum test
##
## data: Total_crash by Condition
## Kruskal-Wallis chi-squared = 0.32696, df = 3, p-value = 0.9549
LaneBreaks
##
## Kruskal-Wallis rank sum test
##
## data: Lane_Breaks by Condition
## Kruskal-Wallis chi-squared = 0.23381, df = 3, p-value = 0.972
summary(GlanceTime)
## Df Sum Sq Mean Sq F value Pr(>F)
## Condition 3 280 93.25 0.953 0.424
## Residuals 40 3914 97.86
None of the 6 metrics have significant differences between conditions
To investigate weather differences in demographics have had an influence on conditions a plot of means for each metric will be performed with each independent variable. This will be done with wiskers as 95% confidence interval and if the data has homoscedasticity it could be possible to eyeball significant influences. If these are detected we’ll perform a t-test
Dependent variables:
- SUS
- SART
- Total Crash
- Lane Breaks
- Total Glances
- Total Glance Time
Independent variables:
- Condition
- Gender
- Gaming
- Drivers licence (This plot will have a linear model underneath it
(x=Years with drivers license y=Dependent variable)) - Automatic
gear
- Car Simulator
SaveSixPlots(Metric="SUS")
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
##
## [[5]]
##
## Call:
## lm(formula = y ~ x, data = SartSusDemo)
##
## Residuals:
## Min 1Q Median 3Q Max
## -30.709 -5.248 3.083 9.022 16.419
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 81.9245 4.1078 19.944 <2e-16 ***
## x -0.5573 0.7688 -0.725 0.473
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 12.19 on 42 degrees of freedom
## Multiple R-squared: 0.01236, Adjusted R-squared: -0.01116
## F-statistic: 0.5255 on 1 and 42 DF, p-value: 0.4725
##
##
## [[6]]
##
## [[7]]
SaveSixPlots("SART")
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
##
## [[5]]
##
## Call:
## lm(formula = y ~ x, data = SartSusDemo)
##
## Residuals:
## Min 1Q Median 3Q Max
## -11.1972 -2.7311 0.4113 2.7137 10.0469
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 15.8408 1.4230 11.13 4.15e-14 ***
## x -0.2663 0.2663 -1.00 0.323
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 4.223 on 42 degrees of freedom
## Multiple R-squared: 0.02326, Adjusted R-squared: 1.648e-06
## F-statistic: 1 on 1 and 42 DF, p-value: 0.323
##
##
## [[6]]
##
## [[7]]
### Total crash
SaveSixPlots("Total_crash")
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
##
## [[5]]
##
## Call:
## lm(formula = y ~ x, data = SartSusDemo)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.7467 -0.5342 -0.4567 0.4499 2.5671
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.31767 0.26684 1.190 0.241
## x 0.04767 0.04994 0.954 0.345
##
## Residual standard error: 0.7919 on 42 degrees of freedom
## Multiple R-squared: 0.02123, Adjusted R-squared: -0.002073
## F-statistic: 0.9111 on 1 and 42 DF, p-value: 0.3453
##
##
## [[6]]
##
## [[7]]
SaveSixPlots("Lane_Breaks")
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
##
## [[5]]
##
## Call:
## lm(formula = y ~ x, data = SartSusDemo)
##
## Residuals:
## Min 1Q Median 3Q Max
## -4.543 -2.116 -0.846 1.936 11.433
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 3.2883 1.1018 2.985 0.00472 **
## x 0.1394 0.2062 0.676 0.50263
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 3.27 on 42 degrees of freedom
## Multiple R-squared: 0.01077, Adjusted R-squared: -0.01278
## F-statistic: 0.4572 on 1 and 42 DF, p-value: 0.5026
##
##
## [[6]]
##
## [[7]]
SaveSixPlots("Total_Glances")
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
##
## [[5]]
##
## Call:
## lm(formula = y ~ x, data = SartSusDemo)
##
## Residuals:
## Min 1Q Median 3Q Max
## -19.0630 -7.3500 0.1733 6.0047 31.5646
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 49.0715 3.6618 13.401 <2e-16 ***
## x -0.1862 0.6853 -0.272 0.787
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 10.87 on 42 degrees of freedom
## Multiple R-squared: 0.001754, Adjusted R-squared: -0.02201
## F-statistic: 0.0738 on 1 and 42 DF, p-value: 0.7872
##
##
## [[6]]
##
## [[7]]
### Total Glance time
## [[1]]
##
## [[2]]
##
## [[3]]
##
## [[4]]
##
## [[5]]
##
## Call:
## lm(formula = y ~ x, data = SartSusDemo)
##
## Residuals:
## Min 1Q Median 3Q Max
## -16.2765 -8.3324 0.0377 5.3081 22.1420
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 38.4526 3.3570 11.454 1.67e-14 ***
## x -0.3189 0.6283 -0.508 0.614
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 9.963 on 42 degrees of freedom
## Multiple R-squared: 0.006096, Adjusted R-squared: -0.01757
## F-statistic: 0.2576 on 1 and 42 DF, p-value: 0.6144
##
##
## [[6]]
##
## [[7]]
LaneBreaksAutomatic <- t.test(formula = Lane_Breaks ~ Automatic, data = SartSusDemo)
SartGender <- t.test(formula = SART ~ Gender, data = SartSusDemo)
CrashComputer <- kruskal.test(formula = Total_crash ~ ComputerGames, data = SartSusDemo)
CrashAutomatic <- t.test(formula = Total_crash ~ Automatic, data = SartSusDemo)
GlanceTimeAutomatic <- t.test(formula = Total_Glance_time ~ Automatic, data = SartSusDemo)
GlanceCondition <- kruskal.test(formula = Total_Glance_time ~ Condition, data = SartSusDemo)
GlancesGender <- t.test(formula = Total_Glances ~ Gender, data = SartSusDemo)
GlancesAutomatic <- t.test(formula = Total_Glances ~ Automatic, data = SartSusDemo)
#MÃ¥ske er der noget med glances og condition
attach(SartSusDemo)
GlancesCondition <- pairwise.t.test(Total_Glances, Condition,p.adj = "bonf")
detach()
LaneBreaksAutomatic
##
## Welch Two Sample t-test
##
## data: Lane_Breaks by Automatic
## t = 2.6531, df = 17.133, p-value = 0.01666
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 0.618242 5.406572
## sample estimates:
## mean in group No mean in group Yes
## 6.076923 3.064516
SartGender
##
## Welch Two Sample t-test
##
## data: SART by Gender
## t = -2.4169, df = 17.261, p-value = 0.02699
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -5.9791715 -0.4090638
## sample estimates:
## mean in group F mean in group M
## 12.10000 15.29412
CrashComputer
##
## Kruskal-Wallis rank sum test
##
## data: Total_crash by ComputerGames
## Kruskal-Wallis chi-squared = 2.116, df = 2, p-value = 0.3471
CrashAutomatic
##
## Welch Two Sample t-test
##
## data: Total_crash by Automatic
## t = 1.9435, df = 13.821, p-value = 0.0726
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.06767813 1.35800071
## sample estimates:
## mean in group No mean in group Yes
## 1.0000000 0.3548387
GlanceTimeAutomatic
##
## Welch Two Sample t-test
##
## data: Total_Glance_time by Automatic
## t = 2.953, df = 23.453, p-value = 0.007043
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## 2.626666 14.871746
## sample estimates:
## mean in group No mean in group Yes
## 43.09308 34.34387
GlanceCondition
##
## Kruskal-Wallis rank sum test
##
## data: Total_Glance_time by Condition
## Kruskal-Wallis chi-squared = 2.4534, df = 3, p-value = 0.4838
GlancesGender
##
## Welch Two Sample t-test
##
## data: Total_Glances by Gender
## t = 1.8374, df = 24.153, p-value = 0.07848
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.6707051 11.5883521
## sample estimates:
## mean in group F mean in group M
## 52.40000 46.94118
GlancesAutomatic
##
## Welch Two Sample t-test
##
## data: Total_Glances by Automatic
## t = 1.8943, df = 20.358, p-value = 0.07249
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.6836275 14.3610468
## sample estimates:
## mean in group No mean in group Yes
## 53.00000 46.16129
GlancesCondition
##
## Pairwise comparisons using t tests with pooled SD
##
## data: Total_Glances and Condition
##
## Acrylic Bar Combination
## Bar 1.0 - -
## Combination 0.2 0.5 -
## Touch_only 1.0 1.0 0.2
##
## P value adjustment method: bonferroni
When analysing this much data the risk of p-hacking increases. We’ll only use these significanses if there is a big heteroscedasticity in regards to the four conditions maybe this change can explain
The four significant effects were:
1) Lane_Breaks by Automatic 0.01666
2) Total_Glance_time by Automatic 0.007043
AutomaticExperienceCondition
Automatic experience is distributed quite nice with a difference of 2
between highest and lowest. Therefore this effect is irrelevant.
3) SART by Gender 0.02699
GenderCondition
Gender is distributed with a difference of 1 between highest and lowest.
Therefore this effect is irrelevant